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GRAPHICAL USER INTERFACE UTILIZING THREE-DIMENSIONAL 
SCATTER PLOTS FOR VISUAL NAVIGATION OF PICTURES IN A 

PICTURE DATABASE 

5 FIELD OF THE INVENTION 

The present invention relates to graphical user interfaces in general, 
and in particular to graphical user interfaces adapted for browsing and retrieving 
pictures in picture databases. 



10 BACKGROUND OF THE INVENTION 

Digital pictures and digital picture albums/collections are prolifer- 
ating with recent advances in quality and cost reductions. Digital pictures can be 
stored in common or individual databases that allow a user to retrieve his or her 
pictures for viewing and reproduction via a computer system. The pictures can be 

15 stored as discrete files, or as linked files such as in a strip of film, for example. 
When a large number of pictures are stored in a collection, it is desirable to be 
able to browse and retrieve selected pictures with a graphical user interface (GUI). 
Used for such an application, a GUI conveniently organizes and groups database 
pictures, and allows a user to browse the pictures in one or more displays. 

20 Along with the digital information representing pictures, digital 

image files can also contain metadata, or information not displayed, which none- 
theless characterizes the data comprising the pictures. The metadata can describe 
the scene in the picture, such as a caption, and can also provide in a straightfor- 
ward manner, information such as the date and time the picture was captured, the 

25 location from which the picture was captured, identify people or objects in the 
picture, and information regarding format and data structure. 

Many prior art digital cameras can be programmed to automatically 
store along with the actual image, the date and time a picture was captured. More 
advanced digital cameras can also be programmed to automatically store along 

30 with the actual image, the location of picture capture by harnessing automatic lo- 
cation systems. For example, the Global Positioning Satellite (GPS) is a well- 
known method for pinpointing the location of a special GPS receiver with a fairly 
high degree of accuracy. Other methods include the use of Radio Triangulation 
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(RT) systems. Using such an approach, a GPS receiver can be either incorporated 
in the hardware of the digital camera, or located nearby. A subsequent image file 
will contain not only the raw image data, but also a date and time stamp, along 
with header information related to the location of the GPS receiver when the im- 
5 age is collected. 

Still more advanced digital cameras may contain pattern recogni- 
tion software for identifying objects and people in an image, and converting such 
information to metadata. 

Whether metadata is entered automatically by the camera, manu- 
10 ally via the camera during picture captures, or subsequently into the digital image 
database, there remains a great need for improved GUIs for efficiently and attrac- 
tively arranging pictures in a digital image database. 

Many prior art methods for navigating picture databases still in- 
volve the use of Boolean word searches, which can be both awkward and unpleas- 
15 ant to view. Even when prior art GUIs are used for picture database navigation, it 
is desirable to browse an entire filmstrip without disregarding pictures that do not 
match search criteria. It is also desirable to search via icons representing pictures 
without having to type in search terms, and to have the ability to dynamically 
change a GUI display and search criteria with "drag and drop" operations. 

20 

SUMMARY OF THE INVENTION 

To overcome the substantial deficiencies associated with prior art 
picture database navigation and retrieval operations, the method of the present in- 
vention at least includes the steps of providing a first display dimension repre- 

25 senting a first characteristic of grouped pictures, providing a second display di- 
mension representing a second characteristic of grouped pictures, providing a 
third display dimension representing a third characteristic of grouped pictures, and 
providing in the manner of a scatter plot, a plurality of picture icons plotted ac- 
cording to a first dimension along a first axis corresponding to the first display 

30 dimension, and a second dimension along a second axis corresponding to the sec- 
ond display dimension, each picture icon representing a group of pictures in a 
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picture database. The third dimension is evidenced by distinct visual characteris- 
tics of the icons. 

The present invention also provides a graphical user interface 
adapted to navigate and retrieve pictures in a picture database. The graphical user 

5 interface at least includes a first display dimension generator adapted to generate a 
first display dimension representing a first characteristic of grouped pictures, a 
second display dimension generator adapted to generate a second display dimen- 
sion representing a second characteristic of grouped pictures, a third display di- 
mension generator adapted to generate a third display dimension representing a 

10 third characteristic of grouped pictures, and an icon generator adapted to generate 
in the manner of a scatter plot, a plurality of picture icons plotted according to a 
first dimension along a first axis corresponding to the first display dimension, and 
a second dimension along a second axis corresponding the second display dimen- 
sion, each picture icon representing a group of pictures in a picture database. The 

15 third dimension is evidenced by distinct visual characteristics of the icons. 



BRIEF DESCRIPTION OF THE DRAWINGS 

Features and advantages of the present invention will become ap- 
parent to those skilled in the art from the description below, with reference to the 

20 following drawing figures, in which: 

Figure 1 is an example of a display generated by the graphical user 
interface (GUI) of the present invention, wherein the axes of three-dimensional 
scatter plot contained therein, represent "who," "what," and "when" picture cate- 
gorization information; 

25 Figure 2 is an example of a display generated by the GUI of the 

present invention, wherein the axes of three-dimensional scatter plot contained 
therein, represent "what," "when," and "where" picture categorization informa- 
tion; 

Figure 3 is an example of a display generated by the GUI of the 
30 present invention, wherein the axes of three-dimensional scatter plot contained 

therein, represent "when," "where," and "why" picture categorization information; 
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Figure 4 is an example of a display generated by the GUI of the 
present invention, wherein the axes of three-dimensional scatter plot contained 
therein, represent "who," "when," and "where" picture categorization information; 
and 

5 Figure 5 is a general schematic block diagram of a system capable 

of implementing the present-inventive GUI. 



DETAILED DESCRIPTION OF THE INVENTION 

The graphical user interface (GUI) of the present invention novelly 

10 uses a dynamically linked, three-dimensional scatter plot for navigating and re- 
trieving pictures in picture database, based upon picture metadata of pictures and 
picture filmstrips. The first two dimensions are displayed by locating icons (each 
representing a group of pictures) in two dimensions according to an ordered pair. 
The abscissa and ordinate pertain to two separate picture characteristics. 

1 5 The third dimension of the three-dimensional scatter plot is a third 

picture characteristic. The value of the third dimension is indicated by the visual 
appearance of the plotted icons. In the preferred embodiment, the visual appear- 
ance and hence the third dimension is indicated by distinct colors which can be 
assigned to specific gradations of the third picture characteristic. However, other 

20 approaches to indicating the third dimension are possible, given the essence of the 
present invention, including unique shapes of the icon. 

Referring to Figure 1, the three-dimensional scatter plot 100 shows 
one embodiment of the present-inventive GUI. The axes (102 and 104) of the 
three-dimensional scatter plots of the present-inventive GUI represent picture 

25 characteristics such as "Who," "What," "When," "Where," and "Why." "Who" 
refers to people in the pictures identified by the metadata. "What" refers to dis- 
tinct events identified by the metadata. "When" is a temporal characterization of 
the pictures and can include the date and/or the time of capture, as identified by 
the metadata. "Where" refers to the capture location of the pictures. "Why" re- 

30 fers to specific comments or explanations about the pictures in the metadata. Al- 
most any standard comments can be accommodated by the GUI of the present in- 
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vention. For example, the comments supported can include the identity of the 
photographer where one can be identified. 

The Who- What- When display 100 in Figure 1 has the ability to 
group pictures in the database according to the people featured, according to 
5 events portrayed by the pictures, and by the year-date the pictures were captured. 
The icons 108-126 represent picture groupings in the scatter plot. Since the 
drawing figures are black and white, the colors are represented by the patterns on 
the icons, according to standard drafting conventions. The red (or pink) icons rep- 
resent pictures captured in the year 2002; brown in the year 2001 ; blue in the year 
1 0 2000; gray (or silver) in the year 1999; violet (or purple) in the year 1998; green in 
the year 1997; orange in the year 1996; and yellow (or gold) in the year 1995. 
Additional years, and of course distinct colors can be used with the present inven- 
tion. 

The icon 108 can represent a group of pictures in the database that 
1 5 contain a first identifiable person (e.g., Husband/Father of a family, or 

Wife/Mother), as part of a first identifiable event (e.g., New Year's Day), captured 
in the year 1999. As another example, the icon 116 might represent a group of 
pictures in the database containing the first-born daughter during a fourth identifi- 
able event (e.g., Valentine's Day), captured during the year 2000. 
20 The user can browse thumbnail representations of the pictures in a 

group by activating the icon (e.g., clicking a pointer while it is placed over the top 
of the icon). 

The three-dimensional scatter plot 200 in Figure 2 is an example of 
a What- When- Where display using identifiable events and dates as the abscissa 

25 and ordinate (along the axes 202 and 204), respectively. The color (or other dis- 
tinguishing appearance) of the icons 208-230 represents the third dimension, 
which is the location of capture. As one example, there is a group of pictures rep- 
resented by the icon 230 in the example database which depicts a fifth identifiable 
event, captured on a fourth identifiable date (this can be general, such as a year), 

30 and captured at a seventh identifiable location indicated by an orange color. 
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From the scatter plot 200 in Figure 2, it can also be seen that at 
both the second and third identifiable locations, there are pictures which depict a 
third identifiable event captured on a second identifiable date. 

The display 300 in Figure 3 is an example of a When-Where-Why 
5 three-dimensional scatter plot with axes 302 and 304, and picture group icons 
308-326. Analogous to the scatter plots of Figures 1 and 2, the scatter plot 300 
contains groups of pictures whose coordinates are the date of capture, the location 
of capture, and one of a predefined number of comments or explanations (in the 
metadata). 

10 The display 400 in Figure 4 is an example of a Who- When- Where 

three-dimensional scatter plot with axes 402 and 404, and picture group icons 
408-430. Analogous to the scatter plots of the preceding drawings figures, the 
scatter plot 400 contains groups of pictures whose coordinates are an identifiable 
person or persons depicted in the pictures, the date of capture, and the location of 

15 capture. As with all scatter plots of the present invention, the coordinates are de- 
termined by the picture metadata. 

In an alternate embodiment, the icons could be in the nature of 
spoked wheels, with the number of spokes indicating a different value of the third 
characteristic. 

20 It will be appreciated by those skilled in the art, that many more 

combinations of "Who," "What," "When," "Where," and "Why" information can 
be used to generate scatter plots with the present-inventive GUI. It will also be 
appreciated by those skilled in the art, that other information categories based on 
metadata can be used to represent the three dimensions of a scatter plot, given the 

25 description of the present invention. 

Given the disclosure of this Letters Patent, those skilled in the art 
will be able to implement a GUI which generates a three-dimensional scatter plot 
for more efficient and user- friendly browsing and retrieval of pictures stored in an 
image database. However, a general GUI system 500 for implementing the pres- 

30 ent invention is conceptually illustrated in Figure 5. 

The various components of the system 500 need not have physical 
proximity. Indeed, the system 500 can be self-contained in a stand-alone com- 
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puter system, part of a Local Area Network (LAN), or part of a remote processing 
system using Wide Area Networks (WANs) such as the Internet, to name a few 
variations. 

The system 500 includes a CPU 502 as do computer systems in 
5 general, a user interface 504 which allows a user to input commands and informa- 
tion (e.g., via a keyboard and mouse), and a display 506 for viewing the scatter 
plots, thumbnails, pictures, and other graphics, and a GUI generator 508 for gen- 
erating dynamic scatter plots. 

The GUI generator 508 generates the scatter plots representing 
1 0 pictures in the picture database loaded into a GUI memory 510. A GUI control 
512 strips the metadata from the image files to determine the scatter plot to be 
generated, along with the value of the three dimensions. In response to the GUI 
control 512, a graphics generator 514 generates for display, the axes, icons and 
other graphics to produce the present-inventive, picture database three- 
1 5 dimensional scatter plots. 

The present invention allows the user to specify exactly which 
picture characteristics to use for the scatter plots, and which dimension to corre- 
spond to a given picture characteristic. A change or update in one dimension 
automatically updates the other two picture dimensions. 

20 

Variations and modifications of the present invention are possible, 
given the above description. However, all variations and modifications which are 
obvious to those skilled in the art to which the present invention pertains are con- 
sidered to be within the scope of the protection granted by this Letters Patent. 



25 
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PARTS LIST 

1 00 Three-dimensional scatter plot (who-what-when) 

102 Horizontal axis (who) 

104 Vertical axis (what) 

1 08- 1 26 Picture group icons (when) 

200 Three-dimensional scatter plot (what-when-where) 

202 Horizontal axis (what) 

204 Vertical axis (when) 

208-230 Picture group icons (where) 

300 Three-dimensional scatter plot (when- where- why) 

302 Horizontal axis (when) 

304 Vertical axis (where) 

308-326 Picture group icons (why) 

400 Three-dimensional scatter plot (who-when-where) 

402 Horizontal axis (who) 

404 Vertical axis (when) 

408-430 Picture group icons (where) 

500 Graphical User Interface (GUI) system 

502 Central Processing Unit (CPU) 

504 User interface 

506 Display 

508 GUI generator 

510 GUI memory 

512 GUI control 

514 Graphics generator 



